Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[confmap] Remove top level condition when considering struct as Unmarshalers. #9862

Merged
merged 19 commits into from May 23, 2024

Conversation

atoulme
Copy link
Contributor

@atoulme atoulme commented Mar 27, 2024

This is a slice of #9750 focusing on removing the top level condition on unmarshaling structs.

@atoulme atoulme requested a review from a team as a code owner March 27, 2024 06:08
@atoulme atoulme changed the title Remove top level block Remove top level lock when considering struct as Unmarshalers. Mar 27, 2024
@atoulme atoulme changed the title Remove top level lock when considering struct as Unmarshalers. [confmap] Remove top level lock when considering struct as Unmarshalers. Mar 27, 2024
Copy link

codecov bot commented Mar 27, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 91.86%. Comparing base (ff7a485) to head (3958649).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #9862      +/-   ##
==========================================
- Coverage   91.88%   91.86%   -0.03%     
==========================================
  Files         360      360              
  Lines       16728    16732       +4     
==========================================
  Hits        15371    15371              
- Misses       1020     1022       +2     
- Partials      337      339       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

confmap/confmap_test.go Show resolved Hide resolved
confmap/confmap.go Outdated Show resolved Hide resolved
confmap/confmap_test.go Show resolved Hide resolved
@atoulme
Copy link
Contributor Author

atoulme commented Apr 1, 2024

Copy link
Member

@mx-psi mx-psi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I finally understood this! I filed #9879 separately that adds two confmap tests. I would want to merge that one and #9861 first if that makes sense to you.

confmap/confmap.go Outdated Show resolved Hide resolved
confmap/confmap.go Outdated Show resolved Hide resolved
confmap/confmap.go Outdated Show resolved Hide resolved
confmap/confmap.go Outdated Show resolved Hide resolved
confmap/confmap.go Outdated Show resolved Hide resolved
confmap/confmap.go Outdated Show resolved Hide resolved
confmap/confmap_test.go Show resolved Hide resolved
confmap/confmap.go Outdated Show resolved Hide resolved
@mx-psi
Copy link
Member

mx-psi commented Apr 2, 2024

It would also be interesting to try and reproduce the failing contrib test on a unit test here, so we can work on it. The offending Unmarshal function is here https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/e4c5b51ec0e24ca163bcb64d944455577f90a09f/pkg/stanza/operator/helper/time.go#L49

mx-psi added a commit that referenced this pull request Apr 2, 2024
**Description:** Adds two tests to confmap to test some edge cases

**Link to tracking Issue:** Written while reviewing #9862
@atoulme
Copy link
Contributor Author

atoulme commented Apr 3, 2024

It would also be interesting to try and reproduce the failing contrib test on a unit test here, so we can work on it. The offending Unmarshal function is here https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/e4c5b51ec0e24ca163bcb64d944455577f90a09f/pkg/stanza/operator/helper/time.go#L49

added a skipped test here: TestRecursiveUnmarshaling

@atoulme
Copy link
Contributor Author

atoulme commented Apr 3, 2024

If you want a foolproof solution, we might have to do something a little more drastic. We might need to keep a hashset of all pointers of unmarshaler methods ever called, to make sure we don't run into cycles.

@mx-psi
Copy link
Member

mx-psi commented Apr 3, 2024

If you want a foolproof solution, we might have to do something a little more drastic. We might need to keep a hashset of all pointers of unmarshaler methods ever called, to make sure we don't run into cycles.

I guess we can forbid what the stanza package does instead. Do you think there is a benefit to being able to do what stanza does? I can't see any advantage right now

@atoulme
Copy link
Contributor Author

atoulme commented Apr 4, 2024

No, I don't see any advantage, except maybe if you're trying to unmarshal and set all values on the struct atomically. Even then, this is a corner case. I think we're ok.

confmap/confmap_test.go Outdated Show resolved Hide resolved
Copy link
Member

@mx-psi mx-psi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am okay with the current approach, I guess we should document the limitation about nested unmarshalling before merging, but I think we can go with this.

cc @open-telemetry/collector-approvers, I would appreciate a second pair of eyes since this is somewhat subtle. Even if this is ready before, I will wait at least until Monday to merge to give others some time to review.

@atoulme
Copy link
Contributor Author

atoulme commented Apr 21, 2024

One more PR in contrib needed: open-telemetry/opentelemetry-collector-contrib#32577

djaglowski pushed a commit to open-telemetry/opentelemetry-collector-contrib that referenced this pull request Apr 22, 2024
**Description:**
Update contrib to match core with the latest changes that are needed for
open-telemetry/opentelemetry-collector#9862
@atoulme
Copy link
Contributor Author

atoulme commented Apr 25, 2024

The last contrib issue is solved by open-telemetry/opentelemetry-collector-contrib#32660. Unfortunately, I couldn't find a good way to fix this - in effect, a breaking change.

Copy link
Contributor

@evan-bradley evan-bradley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, I think this is probably the cleanest solution given the current state of things.

confmap/confmap.go Outdated Show resolved Hide resolved
Copy link
Member

@mx-psi mx-psi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM after adressing Evan's comment. This will need an update on contrib since it's breaking the tests there

confmap/confmap_test.go Show resolved Hide resolved
.chloggen/remove_top_level_block.yaml Outdated Show resolved Hide resolved
atoulme and others added 10 commits April 30, 2024 16:41
Co-authored-by: Pablo Baeyens <pbaeyens31+github@gmail.com>
Co-authored-by: Pablo Baeyens <pbaeyens31+github@gmail.com>
Co-authored-by: Pablo Baeyens <pbaeyens31+github@gmail.com>
@atoulme atoulme changed the title [confmap] Remove top level lock when considering struct as Unmarshalers. [confmap] Remove top level condition when considering struct as Unmarshalers. May 3, 2024
.chloggen/remove_top_level_block.yaml Outdated Show resolved Hide resolved
@codeboten
Copy link
Contributor

@atoulme are these expected:

    config_warnings_test.go:92: 
        	Error Trace:	/tmp/opentelemetry-collector-contrib/exporter/datadogexporter/config_warnings_test.go:92
        	Error:      	elements differ
        	            	
        	            	extra elements in list B:
        	            	([]interface {}) (len=1) {
        	            	 (string) (len=125) "\"metrics::histograms::send_count_sum_metrics\" has been deprecated in favor of \"metrics::histograms::send_aggregation_metrics\""
        	            	}
        	            	
        	            	
        	            	listA:
        	            	([]string) (len=1) {
        	            	 (string) (len=125) "\"metrics::histograms::send_count_sum_metrics\" has been deprecated in favor of \"metrics::histograms::send_aggregation_metrics\""
        	            	}
        	            	
        	            	
        	            	listB:
        	            	([]string) (len=2) {
        	            	 (string) (len=125) "\"metrics::histograms::send_count_sum_metrics\" has been deprecated in favor of \"metrics::histograms::send_aggregation_metrics\"",
        	            	 (string) (len=125) "\"metrics::histograms::send_count_sum_metrics\" has been deprecated in favor of \"metrics::histograms::send_aggregation_metrics\""
        	            	}
        	Test:       	TestSendAggregations/metrics::histograms::send_count_sum_metrics_set_to_false

@mx-psi
Copy link
Member

mx-psi commented May 8, 2024

@atoulme So that we can merge this soon and address @codeboten's comment, would you be able to do the following?

  1. File a draft PR on opentelemetry-collector-contrib that pulls in this version of opentelemetry-collector + adds the necessary fixes in contrib. This is to 'prove' that we can satisfactorily fix the issue in contrib
  2. Post the PR from (1) here. Once this is done, we can merge this PR.
  3. Update contrib and mark the PR from (1) as ready to review with only the fixes

rimitchell pushed a commit to rimitchell/opentelemetry-collector-contrib that referenced this pull request May 8, 2024
…2577)

**Description:**
Update contrib to match core with the latest changes that are needed for
open-telemetry/opentelemetry-collector#9862
@atoulme
Copy link
Contributor Author

atoulme commented May 10, 2024

Here is the PR open-telemetry/opentelemetry-collector-contrib#32660

It is past midnight, I will follow up tomorrow on it.

@atoulme
Copy link
Contributor Author

atoulme commented May 10, 2024

The contrib PR shows the changes combined with test changes to contrib will work. I think we have reached step 2. I have reached out to Alex to make sure he's ok with this.

jlg-io pushed a commit to jlg-io/opentelemetry-collector-contrib that referenced this pull request May 14, 2024
…2577)

**Description:**
Update contrib to match core with the latest changes that are needed for
open-telemetry/opentelemetry-collector#9862
@atoulme
Copy link
Contributor Author

atoulme commented May 14, 2024

@codeboten are you ok with this approach and the steps I have followed, as well as the checks they provide? Can we merge this PR?

@atoulme
Copy link
Contributor Author

atoulme commented May 17, 2024

Can we agree to merge this by end of next week unless someone objects?

@mx-psi
Copy link
Member

mx-psi commented May 17, 2024

I'll put a reminder to merge in a wekk, @codeboten feel free to merge before then if you are okay with it :)

@mx-psi mx-psi merged commit a8580db into open-telemetry:main May 23, 2024
38 of 48 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

None yet

5 participants